Artificial Neural Network based Stereo Matching in Stereo Vision System Undergraduate Research Activity

Intelligent Space is a space, which has distributed sensory intelligence and it is equipped with actuators. The various devices of sensory intelligence cooperate with each other autonomously yielding high intelligence in the space. Each intelligent agent in the Intelligent Space knows its role and can support human beings. The most important source of information in Intelligent Space are cameras. They provide a dynamic image from multiple views within the space. One way of revealing spatial information of the scene is the use of stereo vision, which transforms the information of two plane images into a 3D description of the scene and recovers depth information in terms of the exact distance. To achieve stereo vision, the corresponding pixels on the two images have to be found. This process is referred to as stereo matching, which is still a challenging task. In the present work we propose a stereo vision system, which is composed of four components. The first component consists of two video cameras with a capture card, the second is a special mounting which is able to rotate the cameras. The capture card is used to transfer the images from the cameras into a PC. The third component is a pure artificial neural network (ANN) to determine the direction in which the cameras have to be rotated. The fourth software component is a motor control subsystem that uses the serial port of the PC and a microcontroller to drive the servo motor on the mounting to adjust the cameras in the appropriate orientation. The camera mounting has been designed to facilitate the adjustment of the cameras into any directions. It includes a servo motor that drives the cameras into the desired orientation. The heart of the system is the ANN which was inspired by the human vision system, where the image from the eyes arrive via the optic nerves into the brain. In the brain the optic nerves are crossed in a complex neural network called the optic chiasm, the output of which controls the movements of the eyes to envision the same spot. The main contribution is a pure ANN that receives two image parts from the cameras and on it’s output it provides a control signal to the servo motor, just like the human brain to the muscles that move the eyes. No conventional computational tools (PC) is required if the network is implemented in a neural hardware. In the lack of such a hardware, a PC simulation has been used to test the system. The motor control subsystem sends out the control parameters received from the ANN to a microcontroller, which then provides the required control signals for the servo motor. The proposed system provides a fast and robust method for stereo matching. In the future it will be embedded on a robot with two on-board cameras. The robot belongs to the above described Intelligent Space. It can also be used in fixed camera systems of the Intelligent Space.